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METHODS AND APPARATUS FOR CONTEXT AND 
EXPERIENCE SENSITIVE PROMPTING IN VOICE APPLICATIONS 

Field of the Invention 

The present invention relates generally to improvements in automated voice response 
systems. More particularly, the invention relates to advantageous systems and techniques for 
providing menus and other assistance to a user of a voice response system. 

Background of the Invention 

Automated voice response systems, particularly systems employing speech recognition, 
must frequently provide guidance to a user so that the user may properly format his or her inputs 
and repeat inputs that were not properly recognized by the system. In order to achieve user 
satisfaction, voice response systems must provide prompts to users that indicate the format and 
content of the inputs needed from a user, so that the user's inputs can be interpreted accurately. 
Li addition, presentation of prompts by the systems must not be too time consuming for a user. 
The nature of voice response systems makes them particularly prone to consume too much of a 
user's time because much of the information presented by a system, particularly menu 
information, is presented in sequence. The information a user needs to hear may be preceded by 
other information of no interest to the user. 

One approach to providing user menus that are not excessively burdensome is the use of 
contextual menus. Contextual prompting provides prompts that are based on the user's position 
in the application. For example, if a user is listening to voicemail, one set of prompts is 
provided. If a user is searching a directory in order to place a call, another set of prompts is 



provided. Such directed selection of prompts helps reduce the number of choices presented to 
the user, but typical present day systems do not sufficiently advantageously distinguish between 
users. For example, expert users may know all the different inputs that are required for each 
stage of an application, othw users may be very experienced with some portions of the 
application but not with other portions, and relatively inexperienced users are likely to be 
unfamiliar with the inputs required for a system and need fairly extensive information, such as a 
relatively complete list of available input choices. 

In typical prior art systems, however, little or no distinction is made between such users. 
Instead, because of the necessity that all users be provided with enough information to enable 
them to provide a correctly formatted input that will achieve the result they desire, all users are 
typically treated in the same manner. More experienced users, therefore, are forced to Usten to 
much more extensive prompts than they need. Systems that provide extensive prompts to all 
users waste time and cause significant dissatisfaction among users that do not require elaborate 
prompts. Conversely, if a system employs shorter prompts, it may run the risk that the prompts 
will be insufficiently detailed for less expert users. 

There exists, therefore, a need for automated voice response systems that are capable of 
determining the experience level of a user for various stages of an application and for providing 
prompts that take into account the particular user's level of experience or ability to successfully 
work with the stage of the appUcation being used. 

Summary of the Invention 

A system according to one aspect of the invention includes a central server hosting 
various modules providing services to users. The modules may suitably employ voice 
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recognition in order to interpret user inputs. When a module needs an input from a user, it 
selects an appropriate prompt for presentation to the user. The module has access to user 
information that includes information indicating the user's experience with each fimction of each 
module. The module examines the user information to determine the user's experience with the 
module and function. Suitably, the module categorizes a user as belonging to an experience 
category, such as novice, intermediate or expert based on the user's level of experience with the 
function. The module selects a prompt associated with the user's level of experience with the 
function and presents it to the user. The use of context and experience information in selection 
of prompts is also discussed in "Methods and Apparatus for Context and Experience Sensitive 

Help in Voice Applications," U.S. Patent Serial No. _/ , assigned to a common 

assignee with the present invention and filed on even date herewith, and incorporated herein by 
reference in its entirety. 

A more complete understanding of the present invention, as well as further features and 
advantages of the invention, will be apparent from the following Detailed Description and the 
accompanying drawings. 

Brief Description of the Drawings 

Fig. 1 illustrates a system employing user and context sensitive prompting according to an 
aspect of the present invention; and 

Fig. 2 illustrates a process of voice response employing user and context sensitive 
prompting according to an aspect of the present invention. 
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Detailed Description 

Fig. 1 illustrates a system 100 employing user and context sensitive prompting according 
to an aspect of the present invention. For purposes of illustration, the system 100 is a central 
telephone system providing telephone services and related services, such as directory lookup, 
voice dialing and voice mail to users. It will be recognized, hov*^ever, that a system such as the 
system 100 may suitably be implemented so as to provide any of a number of services. For 
example, a system may provide bank account information to customers or potential customers or 
maybe used in a customer service center, allowing selection of prerecorded information for some 
problems and referring a customer to an appropriate agent for other problems. 

The system 100 may suitably comprise a central server 102, including a processor 104, 
memory 106 and long term storage 108. The server 102 hosts or has access to information and 
data storage facilities of interest to users, for example, directory information and voice mail 
storage. The directory information may suitably be stored in a directory information database 
110, hosted as a file on the long term storage 108, and the voice mail storage may be allocated to 
storage space 1 12 on the long term storage 108. It will be recognized that storage facilities such 
as the directory 1 10 and the storage space 1 12, and data such as the data stored therein, need not 
be hosted on the server 102, but may be hosted in any desired location or component. 

The system 100 may suitably host a voice response application 1 14, suitably implemented 
as software residing on the long term storage 108 and executed under the control of the processor 
104. The voice response application 1 14 includes a user interface module 1 18A and a plurality 
of additional modules. As illustrated here, the additional modules are a directory module 1 18B, a 
voice dial module 1 18C and a voicemail access module 1 18D. The modules 1 18B, 1 18C and 
1 18D are invoked by the user interface module 1 18A as required, in response to a user's inputs. 



4 



Each user communicates with the server 102 by means of a telephone set such as the 
internal telephone sets 123A. . .123N. Such telephone sets may be part of an internal office 
phone system or may be external sets that connect to the server 102 by dialing a central exchange 
124 communicating with the server 102. When a user makes an initial contact with the server 
102, for example, by placing a call to the central exchange 124 or by picking up a handset of an 
intemal telephone, the user interface module 1 18A provides an initial prompt to the user and 
receives user inputs. Depending on the user's response, the user interface module USA invokes 
the module 1 1 8B, 1 1 8C or 1 1 8D, as appropriate. The module that has been invoked then issues 
prompts, receives user inputs and takes actions, such as placing a call or retrieving voice mail, in 
order to perform the service desired by the user. 

The modules 1 18 A, 1 18B, 1 18C and 1 18D include prompt selection modules 125 A, 
125B, 125C and 1250, in order to provide for selection of prompts appropriate for the operation 
being performed and for the experience level of the user. 

The prompt selection modules 124A, 124B, 124C and 124D maintain awareness of the 
particular function for which prompting is required. During operation of each of the modules 
1 18 A, 1 18B, 1 18C and 1 18D, each module passes information to its prompt selection module 
indicating the active function. Each of the prompt selection modules 124A, 124B, 124C and 
124D receives information indicating the active function, that is, the function with which the user 
is currently engaged, hi addition, each of the prompt selection modules 124A, 124B, 124C and 
124D is operative to receive proficiency information for the user, indicating the user's facility 
with the function being executed and with the use of the system 100 as a whole. The proficiency 
information may suitably be taken fix)m a user information database 130. Each of the prompt 
selection modules 124A, 124B, 124C and 124D uses the proficiency information to select or 



5 



create an appropriate prompt to be provided to the user. Prompts may suitably be selected from a 
prompt repository 132, according to a set of rules implemented by the currently operating one of 
the modules 124A, 124B, 124C and 124D. The prompts may suitably be stored in a prompt 
database 134 stored in the repository 132. 

Alternatively, the currently operating prompt selection module may create a prompt based 
on infonnation stored in the database 134, taking into consideration the module and function 
being employed, as well as proficiency information taken from the database 130. 

Suitably, the prompt delivered to the user is based on the user's proficiency, as indicated 
in a user record stored in the database 130. Each user record may suitably include a fimction 
usage tally for each function, indicating the number of times the user has successfully performed 
the function. Prompts are suitably oriented toward novice, intermediate or expert users. 
Parameters or settings of the prompt selection modules 125 A, 125B, 125C and 125D may 
suitably include criteria for distinguishing between novice, intermediate and expert users and 
may adapt prompts to the appropriate level of user based on the user's proficiency information. 
Alternatively, the prompts stored in the prompt repository 132 may suitably be organized 
according to module, function and fimction usage tally. When a prompt is to be selected, the 
currently operating prompt selection module receives or looks up module, fimction and fimction 
usage tally information, and the currently operating prompt selection module may use this 
information as an index for searching the database 134. 

For example, the database 134 may include a comprehensive explanatory prompt to be 
presented by the user interface module 1 1 8 A to a user with no experience using die system 1 00. 
The prompt might be a comprehensive recitation of all the fimctions that were available, and 
might be associated with a fimction usage tally of 0 for all fimctions. The prompt selection 
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module 125 A would simply search the database 134 for the prompt associated with the user 
interface module 1 18A and also associated with a function usage tally of 0. 

Each of the modules 1 18A, 1 18B, 1 18C and 1 18D has access to the user information 
database 130. The user information database 130 includes a usage history for each user. In the 
embodiment discussed here, the usage history for a user includes a tally for each function, 
indicatmg the number of times the user has successfully employed the function. It will be 
recognized, however, that alternative techniques for describing a user's usage history are 
possible. For example, a proficiency score may be maintained, with the score taking into account 
information including the number of times the user has successfully employed the function as 
well as the number of errors made when using the function and the number of repetitions and 
prompts that have been required. 

The usage history for a user is suitably organized by module. That is, the user's usage 
history for the functions carried out by the module 1 18A are grouped together, the user's usage 
history for the functions carried out by the module 1 18B are grouped together, and so on. This 
organization provides a simple and convenient way for a module to retrieve information relating 
to a user's likely needs for prompting for a particular function. For example, when the voicemail 
module 1 18D is to issue a prompt relating to a fimction, the prompt selection module 125D 
searches the user information database 130 to find a user entry for the user. The voicemail 
module 1 1 8D searches the user entry for the fimction for which the prompt is to be issued. The 
prompt selection module 125D obtains the user's usage tally for the fimction. The prompt 
selection module 125D uses the usage tally as an index to search the prompts related to the 
fimction for which the prompt is to be issued, and to select the prompt associated with the user's 
usage tally value for the fimction. The prompt is then passed to the user interface module 1 18A, 
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which presents the prompt for the user, suitably by playing the prompt using a sound recorder or 
a voice synthesizer. 

The prompts stored in the database 1 34 may suitably be organized by module, by 
functional area and by experience category. The database 134 includes a collection of prompts 
for each of the modules 118A-118D. For example, the voice mail access module 1 1 8D may 
include the functions "play messages," "next message," "delete message," "repeat message" and 
"save message." Depending on a user's facility with each function, different messages may be 
needed to assist the user in providing a correct voice input for a particular function. Thus, for 
example, the database 134 may include three prompts for each of the functions performed by the 
module 1 1 8D. The prompts for each function may include a prompt directed to novice users, a 
prompt directed to intermediate users and a prompt directed to expert users. It will be recognized 
that while three experience categories are described, additional experience categories may be 
employed as desired. 

One example of a prompt might be one directed toward an expert user of the "play 
messages" function of the "voicemail access" module. An expert user might need only a simple 
and short prompt, such as an indication of the function being employed. A suitable prompt for 
such circumstances might be "messages" and might be characterized and indexed as "voicemail 
access, play messages, expert user," or by a suitable symbolic or numerical representation of such 
indexing. 

An exemplary technique of organizing and indexing prompts, that may suitably be 
employed in constructing and searching the database 134, is as follows. The prompts are 
grouped in the database 134 according to the module with which they are associated, and the 
search for a particular prompt is performed in the prompts associated with the module being 
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used. Each module performs a number of different functions, and a number of experience levels 
may be associated with each function. For one embodiment, the prompts for each module are 
organized in the form of an array P(X, g(Fx)), where 

P is the prompt to be played, 

X indicates the function being performed, and 

g(Fx) indicates the experience category associated with the prompt P, where Fx is the 
user's usage tally for the function X. The value of Fx is the number of times a user has 
successfully used the function X. For example, if the function under consideration is the function 
"repeat messages," the value of Fx would be the number of times a user had successfully used 
that function. 

The value of g(Fx) may suitably be an integer and may be assigned a value depending on 
the value of Fx. If a user may have one of m different experience levels for the function X, the 
value of g(Fx) may suitably be defined as follows: 

g(Fx) = lifO <Fx<C, 

g(Fx) = 2ifC, :^x<C2 

g(Fx) = 3ifC2 ^x<C3 

g(Fx) = mifFx>C„v,. 

The values of Ci, C2, C3, and so on, are chosen based on an estimate of how many 
successful uses of a function puts a user into a particular experience category. For example, if a 
novice user is considered to be a user who has successfully used a function four times or less, the 
value of Ci for that function may be set to 5. An intermediate user may be one who has used the 
function 5 to 24 times, in which case the value of C2 is set to 25, and an advanced user may be 
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one who has used the function 25 times or more. In this example, there would be only three 
categories, that is, novice, intermediate and expert, and values would be assigned only to Ci and 
C2. 

To take an example, suppose that a user, Mary, is using the system 100. The user 
information database 130 includes a user entry for Mary. The entry includes function tallies for 
each function, with the function usage tally indicating the number of times the user has 
successfully used the function. The function tallies provide evidence that can be interpreted to 
indicate the user's proficiency for each of the functions performed by the voice dial module 1 18C 
and the voicemail module 1 18D, as well as with the overall system 100, and is organized as 



follows: 

Module Function Function usage tally 

Overall system Main menu accesses 1 88 

Overall system Main menu commands 202 

Voice dialing module Call menu accesses 197 

Voice dialing module Call by contact 98 

Voice dialing module Call by directory 52 

Voice dialing module Call by number 47 

Voicemail module Message menu accesses 5 

Voicemail module Play message 0 

Delete message Voicemail module 0 



The tally for main menu accesses indicates the number of times Mary has employed the 
system 100. Mary's user entry indicates a large number of successful uses of each function of the 
voice dial module 1 18C, suggesting that she is an expert user of the voice dial module 1 18C. 
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Mary's user entry indicates only a few menu accesses of the voicemail module 1 18D and no 
successful uses of the other functions of that module, suggesting that Mary is a novice user of the 
voicemail module 1 18D. Therefore, the voice dial module 1 18C will provide relatively brief 
prompts, and the voicemail module USD will provide more detailed prompts. 

Suppose that Mary accesses the system 100 in order to place a call. Based on her 
expertise with the voice dialing module 1 18C, the dialog between Mary and the system might 
proceed as follows: 

System - "Call control menu." 

Mary - "Dial by directory." 

System - "Directory." 

Mary - "Derek Sanders." 

The prompts are relatively short, and have the advantage of consuming little time. 
Because Mary is an expert user, the prompts are sufficient to allow her the guidance she needs. 
If Mary were a less expert user, the following interaction might take place: 

System - "Call control menu. What would you like to do?" 

Mary - "Dial by directory." 

System - "Directory. Whom would you like to call?" 
Mary - "Derek Sanders." 

Once Mary has finished the interaction, her user tallies for Main menu accesses, main 
menu commands and dial by directory are incremented in order to reflect her additional 
experience. 
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As noted above, Mary's function usage tally suggests that she is less expert with the voice 
mail module 1 18D than with the voice dial module 1 18C. Thus, a dialog with the voice dial 
module 1 18D might proceed as follows: 

System - "Main menu," 

Mary - "Go to messages." 

System - "Message menu. Here you can say "read my messages" or "help" for more 
options." 

Mary - "Read my messages." 

System - "<Plays first messagO. Now you can say, "save message," "delete message," 
or "play next message." At any time during a message, you can say "skip message" in order to 
go to the next message." 

In this case, the initial prompt is brief because Mary is an expert user with respect to the 
initial interaction, that is, with the user interface module 1 18A. The prompts relating to the 
voicemail module 11 8D are more detailed, because Mary is a novice user with respect to the 
voicemail module 118D. 

Fig. 2 illustrates the steps of a process 200 for voice response including user and context 
sensitive prompting according to an aspect of the present invention. At step 202, in response to 
user commands and data inputs, navigation through various modules and functions is performed. 
At step 204, when a prompt is to be presented to a user, function and user data is examined in 
order to provide information required to select an appropriate prompt for the user. The function 
and user data indicates the function for which prompting is required, and the user data indicates 
the user's level of experience with the function and with the voice response system as a whole. 
At step 206, the user data is evaluated and the user is categorized by experience level, for 
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example, novice, intermediate, or expert. At step 208, a prompt matching the function and the 
user's experience category is retrieved from a previously assembled collection of prompts 
indexed by module, function and experience category. At step 210, the prompt is presented to 
the user. At step 212, upon receiving a user input, services are performed and data is entered in 
response to the input. The process then retums to step 202. 

While the present invention is disclosed in the context of a presently preferred 
embodiment, it will be recognized that a wide variety of implementations may be employed by 
persons of ordinary skill in the art consistent with the above discussion and the claims which 
follow below. 
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